(19) 



J 



uiiunimi 



(12) 



(43) Date of publication: 

12.04.2000 Bulletin 2000/15 

(21) Application number: 99119200.6 

(22) Date offing: 07.1 0.1999 



EuropSsches Patentamt 
European Patent Office 
Office europeen des brevets (11) EP 0 993 202 A2 

EUROPEAN PATENT APPLICATION 

(51) Int. a 7 : H04N7/50 



(84) Designated Contracting States: 


(72) Inventor: Ryan, Robert T. 


ATBECHCYDEDKESHFRGBGRIErrULU 


Langhorne, Permsytvanla 19047 (US) 


MCNLPTSE 


Designated Extension States: 


(74) Representative: 


ALLTLVMKROSI 


Scftwabe - Sandmalr - Marx 




Stuntzstrasse 16 


(30) Priority: 09.10.1998 US 169580 


81677 MOnchen(DE) 


(71) Applicant 




MATSUSHITA ELECTRIC INDUSTRIAL CQ, LTD. 




Kadoma-shl, Osaka 571^8501 (JP) 





CM 
< 

CM 
O 
CM 

CO 

o> 
o> 

o 

a. 

LU 



(54) 

(57) An MPEG-2 video signal decoder includes a 
syntax parser which is inrpternentedasastatenrachina 
The state machine defines a plurality of states in which 
discrete parsing operations are performed to decode 
the MPEG-2 bit-stream. A distinct processing time is 
established for each state in the state machine. Even if 
the processing for a particular state is complete before 
the end of the respective processing time for the state* 
the transition from the state to the next state does not 
occur untfl the end of the time interval. The processing 
time for each state is set by a rricroprocessor coupled to 
the state machine. The processing time for each state 



may be changed based on image content or to accom- 
modate changes in the circuitry used to implement the 
state machine. The processing times for the states may 
also be adjusted to accommodate changes in other 
processing elements, separate from the state machine 
but which depend on the state rrtachine for the process- 
ing that they perform. One way m which the processing 
times may be changed is to conform the processing of 
an image or a sequence of images to a predetermined 
maximum time interval. 
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Description 

BACKGROUND OF THE INVENTION 

5 [Q001] The present invention relates to decoders for processing image data which has been cornpressed according 
to a format MPEG-2, specified by the moving pictures experts group (MPEG) and in particular to a decoder in which 
different time intervals may be assigned to the decoding of respective segments of the MPEG-2 bit-stream. 
[0002] Video signaJ compression performed under the MPEG-2 standard is inherently variable rata Video data is 
compressed based on the spatial frequency content of either a sequence of images or on differences in spatial fre- 

10 quency content among the individual images in the sequence, 

[0003] The syntax for the MPEG-2 standard is set forth in International Standard 13818-2 Recommendation ITU-T 
H.262 entitled "Generic Coding of Moving Pictures and Associated Audio Information: Video" and available from 
ISOflEC, Geneva, Switzerland, which is incorporated herein by reference for its teaching of the MPEG-2 video cooing 
standard. This standard defines several layers of data recads which are 

is tto sake tfstnpOcity, the d 

video sequence is represented in several nested layers, the Sequence layer, the Group of Pictures layer, the Picture 
layer, the Slice layer and the Macroblock layer. Each layer record except for the Macroblock record begins with a start 
code that identifies the layer. The record induote header data and p 

[0004] The Sequence layer defines parameters such as picture size and aspect ratio which affect the decoding of 
20 a relatively large number of successive images. The Ooup of Pictures layer defines parameters for a smaller number 
of images and the Picture layer defines parameters for a single image. Inside of an image, a S6ce record defines param- 
eters for a horizontal segment of the image which is (xxrposed of multiple Macrobtocks.. 

[Q005] Thus, the actual image data is i n the Macroblock layer. In a typical MP EG-2 bit-stream, there are many more 
Macroblock and Slice records than Picture, Group of Picture and Sequence records. Accordingly, the headers for the 
25 Sequence records. Group of Picture records and Picture records may be decoded using algorithms implemented in 
software which run on a conve n tional micros 

quentry in the brt-stream and are typically decoded using special purpose hardwara 

[0006] In a typical decoder, the circuitry used to decode the headers of the SGce and Macroblock records is con- 
strained by the circurtry used to If ariy change is rnade in the Macroblock 
30 decorJng process, a corresponding change is typically made to the circuitry which processes the Slice and MaCTobtock 
headers, ft for example, an existing Macroblock decoder is implemented using a semiconductor process technology 
having a smaller geometry (ag. translated from a one micron process to a half-micron process), it may be necessary 
to redesign the hardware that decodes the Slice and Macroblock headers so that the complete bit-stream is processed 
property. 

35 

SUMMARY OF THE INVENTION 

[0007] The present invention is embodied in apparatus for decoding a digitally encoded signal. The apparatus 
includes a state machine which parses header ffrtormation in the encoded signal. The state machine defines states cor- 

40 responding to discrete parsing operations- Each state in the state machine includes a def ined processing time for the 
stata A transition from one state to the next is not irnplemerrted until the end of the defined time regardless of whether 
the actual processing operations performed by the state are complete. The processing time for each state is set by a 
microprocessor coupled to the state machine. These processing times may be changed based on image content or to 
accommodate changes in the circuitry used to implement the state machine. The processing times for the state 

45 changes may ateo be adjusted to accornrnodate changes in other processing elements which are coupled to the state 
machine. \ 

[0008] According to one aspect of the invention, the state machine decodes MPEG-2 encoded image data, parsing 
the MPEG-2 syntax tor the Slice and Macroblock layers. 

[0009] According to yet another aspect of the invention, a predetermined number of decoding cycles are defined for 
so decodng the image and the microprocessor adjusts the assigned processing times to fit the total processing time for 
the image into the predetermined number of decocfing cycles. 

BRIEF DESCRIPTION OF THE DRAWINGS 

55 [0010] 

Figure 1 is a block diagram of an MPEG-2 video decoder which includes an emborjment of the present invention. 
Figure 2 is a block diagram of a VLD decoder suitable for use in the video decoder shown in Figure 1. 
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Figure 3 is a state cfiagram for a slate machine which includes an embodiment of the invention. 

Figure 4 is a ftow-chart diagram which rs useful for describing processing performed by the state machine while it 

is in the SearchO state shown in Figure 3. 

Figure 5 is a flow-chart olagram which is useful for describing processing performed by the state machine white it 
is in the Search! state shown in Figure 3. 

Figure 6 is a flow-chart olagram which is useful for descrixng processing performed by the state machine while it 
is in the Search2 state shown in Figure 3. 

Figure 7 is a ftow-chart olagram which is useful frx describing processing performed by the state machine while it 
is in the Start Code state shown in Figure 3. 

Figure 8 is a flow-chart olagram which is useful tor describing processing performed by the state machine while it 
is in the Slice state shown in Figure 3. 

Figure 9 is a flow-chart dagram which is useful for descnbing processing performed by the state machine while it 

is in the Xsfice Info state shown in Figure 3. 

Figure 10 is a flow-chart olagram which is useful for otesc^ 

is in the Macrobfock Address state shown in Figure 3. 

Figure 11 is a flow-chart dagram which 

is in the Macrobfock Type state shown in Figure 3. 

Figure 12 is a flow-chart dagram which is useful for describing processing performed by the state machine while it 

is in the Macrobfock Quantizer state shown in Figure 3. 

Figure 13 is a flow-chart dagram which is useful for describir^ processing pe^ 

is in the vectors state shown in Figure 3. 

Fip^e 14 is a flowchart cfiagram wh^ 

shown in Figure 13. 

Figure 1 5 rs a flow-chart dagram which is useful for describing processing performed by the state machine while it 
is in the coded block state shown in Figure 3. 

Figure 16 is a flow-chart dagram which is useful for describing processing performed by the state machine whfle it 
is in the Wait state shown in Figure 3. 

Figure 17 is a fk3w-crert dagram which is useful for describing processing performed by the state machine while it 
is in the block state shown in Figure 3. 

Figure 1 8 is a flow-chart dagram which is useful for describi ng processing performed by the state machine while it 
is in the skip state shown in Figure 3. 

Figure 19 is a flow-chart dagram which is useful for describing processing performed by the state machine while it 
is in the end stare shown in Figure 3. 

Figures 20, 21 and 22 are exemplary state transitions performed by the state machine shown in Figure 3 in decod- 
ing an infra-frame encoded Macrobfock, a productively encoded Macrobfock and a bidirectfonally predictivefy 
encoded Macrobfock. respectively. 

DETAILED DESCRIPTION 

[0011] Although the present invention is described in terrre of a syntax parser for an M 

tempteted that it may be practiced with a wide variety of digital data protocols where it is desirable to parse the syntax 
of the protocol as a part of the overaDs^ 
is an example of a protocol for a digital signaL 

[9012] Figure 1 is a block diagram of an exemplary decoder system which embodes the present invention. This 
system includes three components, a decoding processor 110 a rtg>barxiwkfth mem 

essor 1 30. The higM>andwidth memory 1 20 used in the exemplary embodiment of the invention is a RAMBUS memory 
system which is available from NEC and Toshiba. 

[001 3] Briefly, the decoder shown in Figure 1 operates as follows. As a first step; the decoder integrated circuit (IC) 
11 0 is initialized by the microprocessor 1 30 in a particular configuration. If, for example, the decoder is to be used to 
decode 525P signals, the appropriate control values are applied to the internal circuitry of the decocfing processor 1 10 
via the nTjarxxocessor interlace 1 1 2. For the sake of clarity, the control bus between the microprocessor 1 30 and each 
of the elements in the IC 1 1 0 has been omitted. This bus may be, for example, a conventional ft? bus. 
[0014] Once the IC 1 10 has been initialized, the input bit-stream is applied to a parser 116. The parser, described 
below with reference to Figure 2, stores the brt-stream in the memory 1 20. In adcfitfon, the parser identifies start codes 
for the Sequence, Ooup of Pictures and Picture layers and notifies the rriooprocessor 1 30 of the memory locations at 
which these start codes are stored In the exemplary decoder shown in Figure 1, the microprocessor decodes the 
header information for these layers. 

[0015] The DRAM control and address generator 126, controls the RAC interface 170 to store the bit-stream data 
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into the memory 120. fn the exemplary entxxf ment of the invention, a portion of the memory 120 is reserved for use 
as a buffer to ho*d the input bit-stream. This buffer area corresponds to the VBV buffer which is specified in the MPEG- 
2 standard 

[0016] After the bit-stream data is written into the VBV buffer area of memory 1 20, it is read from the memory 1 20 
5 and stored in the buffer memory 121 , also under control of the DRAM control and address generation circuitry 126. The 
memory 121 is a FIFO memory which is filed in bursts from the memory 120 and emptied by the variable length 
decoder (VLD) 1 18 as dictated by the bit-stream syntax 

[0017] The VLD 1 18 parses the SOce and Macroblock layers of the bit-stream to generate blocks of quantized dis- 
crete cosine transform coefficient values. These blocks of values are applied a FIFO memory 128. This FIFO memory 
10 buffers the data between the VLD 118 on the one hand and the Macroblock decoding circuitry 132 on the other hand. 
This memory may also perform the inverse scan function defined in the MPEG-2 standard. The circuitry 132 includes 
an inverse quantizer 1 34 which converts the quantized coeffkaent values into a uniform format and an inverse discrete 
cosine (IDCT) processor converts the frequency ck)^^ 

values. The inverse quantizer 134 and IDCT processor 142 include respective memories 136 and 144 to aid in the 
15 processing operations. The output data provided by the circuitry 132 are blocks of pixel values or drfferentiaJ pixel val- 
ues. 

[0018] Where the values in the blocks are Differential pixel values, they are combined with values from previously 
decoded image frames by a motion compensation processor 161 which includes add and cfip circuitry 158 and 160. 
These circuits, in turn, obtain the image data from the previously decoded frames, which are provided by the half pixel 
20 and interpolation riraitry 1 66. The interpolation circuitry 1 66 obtains the image picture elements (pixels) from the pre- 
viously decocM frames using buffer m 
by the DRAM control and address generation circuitry 126. 

[0019] The decoded image data produced by the motion compensation processor 1 6 1 is stored into buffer memo- 
ries 162 and 164. From the buffer memories 162 and 164 the decoded image data is stored into the memory 120 for 

25 display or for use as reference frame data in decoding motion compensated encoded data from later received image 
fields or frames. Data to be displayed is stored into the memory 1 20 in block format and transferred, in raster-scan for- 
mat to a memory 1 74 to be provided to a display device (not shown) under control of a display processor 1 75. 
[0020] An on-screen display (OSD) processor 171 is also ceiled to provide data to, and to receive data from the 
memory 1 20 via the DRAM control and address generator 1 26 and RAG interlace 1 70. In the exemplary embedment 

30 of the invention, the OSD processor 171 generates user menus and close-caption text which overlays the displayed 
image. 

[0021] As described above, the VLD processor 118 decodes the Slice and Macroblock information in the bit steam 
and passes the decoded, quantized coefficient values to the Macroblock decoding circuitry 1 32. The processing per- 
formed by the circuitry 132 includes the conputatta 
35 values. For high definition images having frame rates of 30 frames per second, it is desirable for this operation to be 
performed as quickly as possible due to the large runrtoerot blocks m a No^ 

ard definition images (La MP@ML) are to be decoded, this operation is not as critical. Thus, depending on the type of 

images which are to be processed, it may be desirable to irnplement the Macroblock decoder using a very fast process 

technology (e.g. Z micron) or using a slower process technology (La 1 micron). 
40 [Q022] Changing the process technology in which the Macroblock decoder is implemented changes the speed at 

which Macrobtocks are processed and thus affects the synchronization of the entire system, ff the VLD processor 118 

is implemented in the same technology as the Macrcttort 

decoders may not change in proportion. Thus, for a Macroblock 

able to totally redesign the VLD processor 118. 
45 [0023] Alternatively and according to the subject invention, the VLD processor 118 may be designed as a state 

machine in which the time required to transition from one state to another may be programmed. Thus, if a change in 

process technology causes tttt 

lock decoder 132, the VLD may be reprograrnmed to increase the time required for selected state transitions in order to 
synchronize the processing of the VLD to that of the Macroblock decoder. 
so [0024] Figure 2 is a block diagram of a variable length decoder 118 according to the present invention. As shown 
in Figure 2, the compressed MPEG 2 video data stream is received from the memory 121 by the shift multiplexer 210. 
The shift multiplexer 210 applies 48 bits of the bit stream to the variable rate processor 212. the VLC tookup table 214 
arxl to the Macrobla* parameter ofe^ 

the Macroblock header. These values are used by the parser 21 2 to convert the variable length coded DCT coefficient 
55 values into fixed length coefficient values which are appSed to the MPEG 2 variable rate parser 212. The values held 
by the VLC lookup table include the Macroblock Addressing values, MacrobJock type, Macroblock pattern and the Mac- 
roblock motion vectors. As the header information tor the various I evete of records is not variable length coded this infor- 
mation may be extracted directly from the bit-steam by the variable rate parser. The decoder 212 is also coupled to a 
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parse rate lookup table 216 which, as described below with reference to Figures 3 through 22, defines parse times for 
each of a set of syntactical items defined in the overall MPEG-2 syntax Each of these syntactical items is parsed by a 
respective state of the state machine irnrjlernerrtation of the parser 21 2 as descnbed below with reference to Figure 3. 
The parse rate lookup table is foaded by the n^cropro^ read by the variable rate parser 21 2. 

5 [0025] As descnbed above, the VLD 18 parses only the Sfice and Macroblcck layers of the MPEG 2 syntax The 
Sequence. Group of Picture, and Picture layers of the syntax are parsed by the microprocessor 1 30. Parameter values 
relevant to the decoding of Slice and MacrobJock records are provided by the microprocessor 130 via the picture 
decode parameters block 21 a 

[0026] The variable rate parser 212 decodes the MacrobJocks into inrjvidual blocks of DOT coefficient values. 
10 These coefficients values and other header items decoded from the Kfecrobiock layers are provided to the Maaobiock 

parameter decoder 220. This decoder provides the OCT coefficients and motion vectors to the MacrobJock decoding 

circuitry 1 32. Control circuitry 222 is coupled to the variable rate parser 21 2 to provide blocks of DCT coefficients to the 

Macrobtock decoder 132 when the decoder 132 asserts the signal READY! 

[0027] Figum 3 is a state diagram of a st 
is This state machine cfivides the decoding of Sto and MacroUocKrecoidsi^ 

in detail below with reference to Figures 4 through 1 9. 

[0028] The 15 defined states include tour states, SearchO. 310; Searchl, 312; Search^ 314; and Start Code, 316. 
These states which locate start codes and, in particular, Slice start codes in the bit steam When a Slice start code is 
identified, the state machine transitions to the SSce state 318. 
20 [0029] Rom the Slice state 318, the state machine transitions to the Xslice Info state 320 and to the MacrobJock 
Address state 322. State 322 begins the deoocfing of the MacrobJock header portion of the bit stream. After the Mar> 
roblock address is calculated at step 322. the state machine tr^ 
from the MacrobJock header information contained m the btt 

state 324, the state machine transition state 326 to retrieve the Macrobtock quantization information and, through state 
25 328, any motion vectors which may exist for the MacrobJock. K, in state 326 it is determined that the MacrobJock is a 
coded MacrobJock. control transitions to state 330 to process the coded MacrobJock. Otherwise, after state 326 and 
after state 330, control transitions to state 332 to waft urrfl the MacrobJock d 
of coefficient values. 

[00301 A MacrobJock may be skipped in two ways. First, in the MPEG-2 syntax, some Macrobiocks are marked as 
so skipped Macrobiocks in the MPEG-2 syntax. These MacrobJock records have header information but no coefficient val- 
ues. The second way in which a MacrobJock may be stopped is if the Macrobtock decoder has encountered an error in 
the bit-stream and is skipping the remainder of the blocks in the slice, the Skip state 334 satisfies the Wait state 332 in 
either of these instances. When the Wait state has been satisfied and the MacrobJock decoder 132 accepts the next 
block of coefficient vatoes, the state machine ti are 
35 decoder 132. After providing to 

exist in this Slice, in which case the state machine transitions to state 322, or whether a new Slice is to be processed 
in which case the state machine transitions to state 31 0. 
[0031] As described above, the microprocessor a^ 

diagram shall occur after the entry into the state. In this irnpternentation, any processing which takes place to a state 
40 occurs in this tinie interval. The actual state 

by that time interval have elapsed. This implementation aliows the VLD processor to be changed adaptivery to process 
shorter or longer item syntaxes rather than being designed to perform the worst case syntax processing for each syn- 
tactical item. 

[0032] For example, under the MPEG standard, a MacrobJock may be defined as a duaHxime MacrobJock in which 
45 case rt may include four motion vectors. An infra-frame coded MacrobJock on the other hand has no motion vectors. 
When the state machine is decoding an irrtra-frame coded MacrobJock, the vector state may be allocated a small or zero 
processing time. When the system is decoding a ciiaHxime MacrobJock, however, twice the normal state processing 
time may be allocated for the vector state to altow the extra rnotx» vectors 

[003?] Figure 4 is a ftow chart diagram which descrtoes the operation of the state machine m state SearchO 310. 

so The first step in this process is step 412, in which the variable rate parser 212 waits until the processing time for the 
previous state has elapsed and accesses the parse rate lookup table 216 for the state SearchO. The value returned 
from table 216 defines a number of system dock pulses which will elapse before there can be a transitjon from state 
SearchO to anclher state of the state in the Search processing is step 414, where the input brt 

stream is analyzed to determine if the next bytes in the bit stream represent a Slice start code value. The start code 

55 values of interest to the subject application are the Sfice start codes which may have values of 0x01 through OxAF 
depenolng on the vertical position of the Slice in the image frame. 

[0034] If a Sfice start code value is found at step 41 4, then the next transition of the state machine is to the Slice 
state 318, descrtoed below with reference to Rgure 8. If a Slice start code value is not found at step 414. the next step 
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in the Sear chO process, step 416, determines if the next bytes in the bit stream correspond to a start code prefix. As 
defined in the MPEG-2 standard, a start code prefix is a string of 23 zero-valued bits followed by a single one-valued 
bit To determine rf a start code prefix has been found in the bit-stream, the state machine courts the number of zeros 
in any group of zeros which are separately parsed by the parser. Thus, in determining rf a start code prefix has been 
5 found in the bit-stream, step 416 determines if the next byte has a value of 1 and rf at least 2 other zero-valued bytes 
were counted immediately before this one-valued byte, rf these conditions are satisfied then a start-code prefix is found 
at step 41 6 and the next state for the state m 

[0035] Al step 418, the Searctt process determines^ 

valued byte, rf so. these zero-valued bytes are removed from the brt-stream, the current count of zero-valued bytes is 
10 stored into a local variable (not shown) of the SearchO state, and the Search2 state 314 is marked as the next state for 
the state machine. Otherwise, step 320 is executed which determines the next byte in the bit stream is a zero. If so, this 
zero-valued byte is removed from the brt-stream, the current zero count value is incremented by one and the next state 
for the state machine is state Search^ 

the bit-stream must be non-zera AccorrJngry, this byte is removed from th e bit-stream and control is transferred to step 
is 412. Essentially this is a transition from state SearchO back upon itself. At step 412. the SearchO state 310 waits until 
the previous allotted time tor state SearchO has elapsed and fetches the new alk^ 
[0036] Rgure 5 is a ftow chart diagram 

51 0 which waits until the allotted time for the previous state has elapsed and fetches the current time for state SearcM 
from the parse rate lookup table 21 6 (shown in Figure 2). After fetching the allotted time for state Searchl , step 51 2 is 

20 executed to determine rf the next bytes in the bit stream correspond to a SOce start code value. If sa control transitions 
to the Sfice state 318. ff a S«k» start code is rxrt found at step 512, stete 514 is executed to determine if the next bytes 
in the bit stream correspond to a start code prefix. H a prefix is found at step 514, the state machine transitions to the 
Start Code state 31a If a start code prefix is not found at step 514, step 516deterrrinesrf the next byte or bytes in the 
bit stream have a value of zera If so, the next state transition for the state machine is to the Search2 slate 314. If, 

2s instead, the next byte in the bit stream is not the next state transition for the state rra 
310. 

[0037] Rgure6isaftowcrtartcfiag/EOT)w 

processes the zero valued stuffing bits which are inserted before certain start codes in the MPEG bit stream. The 
Search2 state 314 begins at step 610 in which the state machine warts until the time allotted for the previous state has 

so elapsed and then fetches the current alkmed time for the Search2 state fr 

control transfers to step 61 2 to determine if the next bytes in the bet stream correspond to a Slice, ff so, the next state 
transition for the state machine is to the Sice state 318. Otherwise, step 61 4 is executed in which the Search2 process 
determines if the next bytes in the bit stream correspond to a start code prefix, rf a start code prefix is encountered at 
step 614, the next state for the state machine is the Start Code state 316. Otherwise, control transfers to step 616 of 

ss the Search2 process. Step 616 determines if the next bytes in the bit stream are zeros. If so, these zero-valued bytes 
are removed from the bit stream and control tr an sfer s to step 610. This is essentially a transition from state Search2 
back upon itself . If the bit-stream includes a long sequence of stuffing brts, state Search2wffl 
itself unto al l of the stufhng bits have been removed from the bit-stream, rf, at step 61 6, the next byte in the bit stream 
is not a 0, the next state transition for the state 

40 Figure 4. 

[0038] Figure 7 is a flew chart cfiagram which aiustrates the operation of a state machine in the Start Code state 
316. Step 710 of this process waits for the prevfou^ 

to the Start Code state from the lookup table 21 a As described above, the Start Code state is entered if a start code 
prefix is found in the bit stream. At step 71 2, the start code prefix is removed from the bit stream and the next bytes of 
46 the bit stream are examined to determine if they correspond to a Slice start code value. If so, the next state transition 
for the state machine s to the Slice state 318. Otherwise, the start code value is not a Sfice start code value and the 
next state for the state machine is the SearchO state 310. 

[0039] Rgure 8 is a flow chart cfiagram which aiustrates the operation of the state machine in the Slice state 31 a 
At step 81 0, the process wafls for the processing of th^ 
so for the Sfice processing from the parse rate lookup table 216. As shown in Table 1 , the syntax cfiagram tor the Slice 
header from the above referenced MPEG-2 speculation, the first item in the Slice header is the quantizer scale infor- 
mation for the Slice. 



55 
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Table 1 
Macroblock 

macroblockO { 

while ( nextbitsO='0000 0001 000') 

macroblock_esc2pe 
mcroblock_address_increment 
macroblock_modesO 
if (macroblock_quant) 

quantiser_scalejx>de 
if (macroblock jnotion jforward 1 1 

(macroblock Jntxa<S^^ 

motion_vectors(0) 
if (macroblock_motion_backward) 

motion_vectors(l) 
if(macTOblockjntnt^ 

marker Jbit 
if (macroblockjmttem) 

coded J>lock_pattern0 
for(i=0;i<blockjcoimt;i+ +){ 
blockCO 

} 

} 

The terms used in the Macroblock syntax are defined in the above referenced 

[0040] At step 812, this information is retrieved from the bit stream and stored. At step 814, the next brt in the brt 
stream is examined to determine if extra information is defined tor this S5ca If so, the Xsiice Info state 320 is the next 
state for the state machine. Otherwise, the next information Mowing the extra bit is the beginning of a Macrobiotic 
Processing of the Macroblock begins with 
[0041] Figure 9 is a ftow chart diagram 

After waiting for the prior state to complete and fetching the currently allotted time for the Xsiice Into state at step 910, 
step 912 is invoked to discard the extra intormatkyi frc^n the brt stream At step 91 4 t the 

diately following the removed extra information record to determine if yet another extra information record is inducted in 
the Slice header. If so, control transfers to step 910. essentially a transition from the Xsiice Info state 320 to itself, to 
remove the next extra information record. Once all of the extra information records have been discarded from the Slice 
header, step 91 4 causes the next state transition of the state machine to be state Macroblock Address 322. 
[0042] Rgure 1 0 is a flow chart cf agram which Blustrates the process performed by state Macroblock Address 322. 
After waiting for the previous s^e to cornplete and obtaining the currency allotted nurnber c# dock cydes tor state Mac- 
roblock Address from the parse rate lookup table 216. the process shown in Figure 10 computes the Macroblock 
address at step 101£ In the MPEG standard, the start code for a Slice is coded to indicate the vertical position of the 
Slice on the reproduced image. In addition, as shown in Table 3, the MPEG-2 syntax for the Macroblock, the first item 
in the Macroblock record is an address increment value; which defines the address of the Macroblock in the Slice. Step 
1012 retrieves the start code value and the address increment value from the bit stream and, using the current SBce 
start code value, calculates the address for the Macroblock in the image. After step 1 01 2, the state transition for the 
state machine is to state Macroblock Type 324. 
[0043] Rgure 11 is a flow chart diagram which mustrales the processing 

After step 1 1 1 0 in which the state machine warts for the prior state to complete and fetches the cunent allotted time for 
the stat e Macroblock Type state, step 1 1 1 2 is executed which fetches the Macrobtock Type field from the Macroblock 
header. 
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[0044] The Macrobiock block type determines whether the motfoncorri^^ 
roblock is frame based or fieW based and whether the DCTcodng of the Macrobtockfe 

step 1114, these values are stored by the slate machine. After step 1114, the next state for the state machine is the 
Macrobiock Quantizer state 326. 

5 [0045] Figure 12 is flow chart diagram shewing the processing performed during the Macrobiock Quantizer state 
326. As described below, this state may be entered several times for each Macrobiock as it contains the state transition 
to the Vector state which extracts the motion vector information from the Macrobiock. To ensure that the Macrobiock 
syntax is interpreted property during the invocations of the state 326 after the first invocation, several Boolean variables 
are defined: Q SCALE, GOT FORWARD, and GOT BACKWARD. The use of these Boolean variables is described 

10 below. 

[0046] At step 121 0, the state machine waits for the previous state to complete and fetches the currently allotted 
time value for the Macrobiock Quantizer state At step 1212, the state machine tests the Boolean state variable Q 
SCALE to determine if the quantizer scale code for this Macrobiock has been obtained. If not then at step 1214, the 
Macrobiock Quantizer process fetches the nex^ 

is and sets the Bcx5lean variable GOT Q SCALE at step 1216. Next at step 121 8 the Macrobkx* Quantizer process tests 
a Boolean variable GOT FORWARD to determirre if the forward motion vectors have been obtained for this Macrobtock. 
If not then at step 1220. the Boolean variable GOT FORWARD is set and the next state for the state machine is set to 
the Vectors state 32a If, however, at step 121 8, the variabte GOT TORWARD was set then corrti^ 
trol 1222 to test the Boolean variable GOT BACKWARD. This Boolean variable is set if the backward motion vectors 

20 have been obtained for the Macrobiock. If they have not control transfers to step 1224 at which the Boolean variable 
GOT BACKWARD is set and the next state of the state machine is set to the Vectors state 328. H at step 1222 the 
Boolean variable GOT BACKWARD was set step 1 226 is executed to determine if a marker bit exists ri the Macrobiock 
header, rf so, step 1 228 is executed to discard the marter bit from the brt stream and control is transferred to step 1 230. 
At step 1230, which is also executed after step 1226 rf the Macrobiock header does not contain a marker bit, the state 

25 Macrobiock Quantizer determine s if the Macrobiock irx^udesacoded block pattern. If so, the next transition for the state 
machine is to the Coded Block state 330. Otherwise, the blocks of the Macrobiock are ready to go to the Macrobiock 
processor 132 (shown in Figure 1) a 

[0047] Figures 13 and 14 are flew chart diagram which describe the operation of the state machine in the Vectors 
state 328. At step 310. the state machine warts for the prior state to complete and fetches the current time aOotted to 

30 the Vectors slate from the parse rate lookup table 21 a As descrtoed above with reference to Figure 1 2, the Vectors 
state is invoked to fetch either a forward motion vector or a backward motion vector. Under the MPEG standard, a Mac- 
robiock may be designated as a predicted Macrobiock having a single forward motion vector, a bidirectjonalry predicted 
Macrobiock. having a forward motion vector and a backward motion vector or a dual-prime Macrobiock in which case it 
has two forward motion vectors, one for each field, and two backward motion vectors. 

ss [0048] Atstep 1312 of the Vectors process, the Macrobiock header is tested to determined 
set of motion vectors or two sets of motion vectors. If the Maoofalock 
step 1314 which fetches the motion vertical fieM select value for to 

transfers c»ntrol to step 1320. Step 1320, invokes the process GET VECT to fetch the actual motion vector ffTformation 
from the Macrobiock header tor the first motionvectorof the two-vector set Thfe 
40 reference to Figure 14. After step 1320. step 1322 is executed which fetches and sara 

value for the second motion vector. After step 1322. the process GET VECT is again invoked at step 1324 to extract the 
motion vector values for the second motion vector of the motion vectw 

ess determines that the Macrobiock contains only a single motion vector, then control transfers to step 1316 which 
determines if the single motion vector includes a rrwtxxi vertical 

45 tion is retrieved from the motion vector header and corrtrol is transferred to step ^ 

tion. After 1 324, the next state for the state machine is the Macrobiock Quantizer state 326. This state is reentered after 
the vector state 328 to ensure that no other motion vectors need to be obtained tor the Macrobiock- Even for a dual- 
prime Macrobiock in a bidirectionalJy encoded image, only two passes through the Macrobiock Quantizer state should 
be needed for a given Macrobiock. 

so [0049] Rgure 1 4 is a flew chart diagram which illustrates the operation of the process GET VECT shown at steps 
1320 and 1324 of Figure 13. The first step in this process, step 1412, fetches the horizontal motion code from the 
motion vector information in the Macrobiock At step 1414. the next bit of motion vector information is fetched from the 
bit stream to determine if residual information exists for this motion vector. rf so. then at step 1 41 6, the residual informa- 
tion is retrieved from the bit-stream and saved. At steps 141 8 and 1 420, the next information in the brt stream is tested 

ss todetemine rf the motion vector isadu^ 1424, 1426, 1 428 arxM 430 repeat the proc- 

ess set forth above except that these steps f^ 

GET VECT. After this step, the process returns to the point m the Vectors state 328 which immecfiatery follows the invo- 
cation of GET VECT 
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[0050] Rgure 15 is a flow chart diagram which illustrates the process performed by the Coded Block state 330 of 
the state machine. At step 1 51 0. the Coded Btock process waits for the previous state to complete and fetches the cur- 
rently allotted time value for the Coded Bock state from the parse rate lookup table 21 a At step 1512, the process 
fetches the coded block inferna l 
s at step 1512, the btocks of the Macrobtock are ready to be r^^ 

the Wait state 332 is trie next state to be entered by the state machina 

[0051] Figure 16 is a flowchart diagram which illustrates the operation of the Wait state 332. At step 1610. the state 
machine waits tor the prior state to complete and fetches the currently allotted time or the Waft state from the parse rate 
lookup table 21 a At step 1612, the wait process determines rf the READY signal provided by the Macrobtock decoder 
10 132 intficates that the decoder 132 is ready to accept another Maoobtodc White 
itself to continually test this signal rxovidedbytrte Macrobtock decoder 132. 
[0052] Once the Macrobtock decoder incficates that it is ready to accept Ma 
ferredtostep1614todeterminerftheM 

skipped in two instances: 1) when the Macrobtock rs marked as being skipped in the MPEG-2 syntax and 2) when the 
is Macrobtock decoder has encountered an error and signals the parser 21 2 to stop the remaining MacroHocks in the 
Slice record. It at step 614, the Macrobtock Stop flag is set then the next state for the state machine is the Skip state 
334. Otherwise, the next state for the stats machine is the Btock state 33a 

[0053] Figure 17 is a flow chart diagram which describes the operation of the Block state 336. At step 1710. the 
pnx»ss determines if the btock that was^ ff so. the next state 

20 for the state machine is set to the End state 33a Otherwise, step 1712 is executed to determine if the Macrobtock 
decoder 1 32 has asserted the REACY signal. The MacroUock decoder 1 32 provides the READY signal when it is ready 
to receive a btock for inverse quantization and inverse cIot 

the block step 1 71 4 rs executed which transfers control to the Macrobtock decoder 1 32. After processing a btock in step 
1714, ccfttrol transfers to step 1710 as de 
25 have been processed. 

[0054] Figure 1 8 is a flow chart diagram which illustrates the operation of the Skp state 334. The Skip state is 
essentially the same as the Btock state except for the step 1814. process s^ 

ing if the Macrcttock was marked as a skipped Macrobtock in the MPEG-2 syntax or 2) removes the bytes correspond- 
ing to the Week to be skipped from the bit steam if the btock is skjaped due to an error in the Macrobtock decod er. This 
30 continues until all of the blocks in the Macrobtock have been processed. After the last block has been processed, step 
1810 transfers control to the Wait state 332. 

[0055] Figure 1 9 is a flow chart dagram which illustrates the operation of the state machine for the End state 338. 
At step 191 0. the state machine waits for the prior state to complete and fetches the current time allotted for the End 
state from the parse rate lookup table 216. At state 1912, the state machine determines if processing tor the current 
ss Slice is complete. If so. control transfers to step 1914 to wait untfl the decoder is ready to process another S6ca After 
step 1 91 4, control is transferred to the SearchO state 31 0 to begin searching for the start code of the next SBce record 
as descrfoed above with reference to Figure 4. 

[0056] If. at step 1912, there are more Macroblocks to be processed in this SSce, then step 1916 is executed to 
determine if the next bytes in the bH stream correspond to a start coda If nr^ 
40 in the Slice and the next state in the state machine is set to the Macrobtock Address state 322, descrtoed above with 
reference to Figure 10. If. at step 191 6 the next bytes in the bit stream etoexxresporri to a start code, then m^ 
robtock in the Slice has been processed and the SearchO state 310 is set as the next state for the state machina 
[0057] Figure 20, 21.and22BlustrateapplicatJCfiso*m^ 

ures shows a sequence of states which is used to decode a Macroblock. Figure 20 shows the exemplary parsing of an 
45 intra-frame or infra-field coded Macrobtock which is the first Macroblock in Hs Slice. Starting at the End state 338. the 
state machine transitions to the SeardiO state 310. then to the Searchl state 312 and then to 
At this point in its processing, the state machine has encountered a start code in the bit stream. Also at state 316, the 
start code is determined to be a Sfice start code causing the next state of the state machine to be the SGce state 31 8, 
in which the Slice header Information is decoded. 
so [Q058] Each of the states 338, 31 0. 312. 316. and 31 8 require a relatively small amount of processing. Accordingly, 
each of these states may be completed in one state cycle time The next state m the process, however, state Macrob- 
lock Address 322 requires more processing time than any of the previous states shown in Figure 20. In the parse rate 
lookup tabte 21 6, this state is aHcrtted twice the state processing time of any of the previous states. This is indicated in 
the drawing figures by the thick border of state 322. Flowing the Macrobtock Address state 322. the state machine 
ss transition to states Macrobtock Type 324. Macrobtock Quantizer 326 and then to the Wait state 332 to wait for the Mac- 
robtock d ecod er 1 32 to accept the blocks of the intracoded Macrobtock. 

[0059] Figure 21 shows the operation of the state machine in d ecoding a prerJctively encoded Macrobtock (P-frame 
or P-field) which is not the fret Macroblock in the Sfica Figure 21 begte 
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roblock Address state 332, as in Figure 20. the End slate is aDo^ one time whTe state MacroWock Address 332 
is allowed two state times After state 322, the state machine transitions to tte 

roblock Quantizer state 326. After 81316326, the state machine transitions to the Vectors state 328. As the processing 
in the Vectors state is also more extensive than in other states, two state processing times are allocated to the Vectors 
state 328. As described above with reference to Figure 1 2 and 1 3, after processing the Vectors state, the state machine 
again returns to the Macroblock Quantizer state to ensure that no other motion vectors need to be obtained. As this is 
a prerictively encoded Macroblock, the bit-stream contains only one motion vector. After the Macroblock Quantizer 
state 326, the state machine transitions to the Coded Block state 330 and then to the Wart state 332. 
[0060] Figure 22 iBustrates the operation of the state machine when processing a bidirectionalry encoded Macrob- 
lock which is also a diial-prime Macroblock. As descrfred above, this Macroblock has four sets of motion vectors. The 
first five states of the state machine shewn in Figure 22 are the same as in Figure 2 

been obtained, however, the Vectors state is invoked a second time to obtain the second set of forward motion vectors 
for the dual-prime Macroblock as descrfoed aibove with reference to Figure 13. Following the second invocation of the 
Vectors state, the state machines transftons 
328 to obtan the first backward moto 

to obtain the second backward motion vector. Following the second invocation of the Vectors slate, the Macroblock 
Quantizer state 326 is again invoked foltowed by the state 332^ 

[0061] If a number of clock cycles, n. is ass^ then 
the syntax processing lor the intracoded Macroblock in Figure 20 

dictivery encoded Macroblock in Figure 21 spans 10 state times and the processing for the bkfrectionaOy encoded Mac- 
roblock shown in Figure 22 spans 16 state times. 

[0062] tn a conventional variable length decoder, the time aDocated the syntax processing of afl Macroblocks in the 
bit-stream would need to conform to the worst case processing (La that shown in Figure 22). By adaptivery allocating 
processing time to the various states, however, a decoder according to the present invention can achieve significant 
time savings in the syntax processing of SI ices and Macroblocks. This time savmgs may allow a slower syntax proces- 
sor to operate with a relatively fast Macroblock decoder J nadcii^ 

erwise vast syntax processor may be slowed down by sirrpry increasing its state delays so that it is corrpatible with a 
slower Macroblock decoder. Thus, a given in te grated circuit (IC) irrplementation of the variable rate parser 212 and 
Macroblock decoder 1 32, may be scaled up or scaled down and any timing variations in the resulting ICs may be com- 
pensated by adjusting the state timing of the parser 212. 
[Q063] While this invention has been descrtoed in terrre of a Macroblock synto 

it is contenplated that it may be practiced with any state machine that processes an input signal to extract information 
which is processed by another processor, tnthrsmarmertherxeixocessor 
of other processing hardware. 

[0064] While the invention has been described in terms of an exemplary embodiment it is contemplated that it may 
be practiced as outlined above within the scope of the appended claims. 

Claims 

1. Apparatus for processing a digitally encxxJed signal c^^ 

multiple record types* each record type including header information and paytoad data, the apparatus comprising: 

a state machine which parses the header information in at least selected ones of the multiple record types, 
wherein the state machine defines states which perform ctiscrete parsing operations and transitions between 
states are determined by parsing the digitally encoded signal; 

means for defiring respective p respective state times 

for a first state and a second state; 

processing means for performing processing steps indicated by the first state; and 
delay means for waiting unta the respective defined processing time for th^ 
to the second state. 

2. Apparatus according to daim 1. further incfudmg, means for adaprjvery specifying the respective processing times 
for each of the state types, wherein the processing time for each state changes 

the records. 

3. Apparatus according tod 

for each of the state types, wherein the processing time for each state changes responsive to changes in other 
processing elements which are coupled to the state machine. 
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4. Apparatus for processing an MPEQ-2 encoded signal, having Sfice level records, each Sfice level record having 
header information and a plurafity of Macroblock level records, each Macroblock level record having header infor- 
mation and a plurality of blocks of frequency-domain coefficient values, the apparatus comprising: 

a state machine including a first state which parses the header information the Sfice level records and a second 
state which parses the header information in the Macroblock level records to extract the plurality of blocks of 
frequency-domain coefficient values from the MPEG-2 encoded signal, the state machine comprising: 

means for defining respective processing times for the first and second states; 
processing means for performing processing steps indicated by the first state; 

delay means for waiting unfl the respective defined processing time for the first state has elapsed to tran- 
sition to the second state; and 

processing means for performing processing steps indicated by the second state to the header information 
in the plurality of Macrob to c te to extract the btocksoi frequency dc^ 
of Macrobtocks; and 

Macroblock decoding means for transforming the extracted blocks of frequency domain coefficient values into 
jsp^flifli domarc^ pixel values; 

& Apparatus accenting to claim 4 wherein the pturafity of Macrobtocte includes precficfively encoded Macrobfocks 
arribkfirectfonallypreo^ 

locks incfaxfing a single motion vector and the bkfirectionaDy predfctively encoded Macrobiocks inducing muftple 
motion vectors, the apparatus further including, means for adoptively specifying the respective processing times for 
each of the first and second states, wherein the processing time for the second state changes responsive to the 
processing of a predictively encoded Macroblock and a bidirectionaJJy predictiveJy encoded Macrocode 

6. Apparatus according to claim 5, wherein a predetermined amount of time is allocated for decoefng a single image 
of the MPEQ-2 data signal and the means for attentively specifying the respective processing times for each of the 
first and second states specifies respective first and second portions of the predetermined amount of time as the 
respective processing times for the first and second states. 

7. Apparatus according to claim 4, further including means for adaptively specifying the respective processing times 
for each of the state types, wherein the processing time for each state changes responsive to changes in the Mac- 
roblock decoding means. 

& A method of processing a digitally encoded signal conforming to a predetermined protocol, the protocol defining 
multiple record types, each record type including header information and paytoad data, the method comprising: 

parsing the header information in at least selected ones of the multiple record types, by performing a discrete 
parsing operation in respective states of a state machine and tr ans itioning between states as the digitally 
encoded signal is parsed; 

deTring respective processing tir^ respective stete times for first one of 

the states and a for a second one of the states; 
performing processing steps indicated by the first state; and 

waiting until the respective defined processing time for the frst state has elapsed to transition to the second 
state. 

9. A method according to claim 8, further including, the step of adaptively specifying the respective processing times 
for each of the state types, wherein the processing time for each state changes responsive to the paytoad data of 
the records. 

10. A method accorrfng to claim 1 further including, the step of adaptively specifying the respective processing times 
for each of the state types, whereto the processi 

performed on the records outside of the stete machine. 

11. A method of processing an MPEQ-2 encoded signal, having Slice level records, each Sfice level record having 
header information and a plurafity of Macroblock level records, each Macroblock level record having header infor- 
mation and a plurality of blocks of frequency-domaki coefficient values, the method comprising the steps of: 
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parsing the header information of the Sfice records in a first state and parsing the header information of the 
Macrobiock a first state which parses the header information the Slice level records and a second state which 
parses the header information in the Macrobiock records in a second state to extract the plurality of btocks of 
frequertt^ctornain coefficient values from the MPEG-2 encoded signal, induding the steps of 

defining respective processing times for the first and second states; 
processing the Sfice header information in the first state; 

waiting until the respective defined processing time for the first state has etepsed to transition to the sec- 
ond state; and 

Processing the Macrobiock header information in the second state to extract the btocks of frequency 
domain coefficient values from the plurality of Macrobiocks; and 

transform i ng the extracted 

12. A method according to claim 1 1 wherein the plurality of Macrobiocks includes pretictivefy encoded Macrobiocks 
andbcfrectionaDypretfc^^ 

locks inducting a single motion vector and the btirectionaDy pretictiveJy encoded Macrobiocks including muttpte 
motion vectors, the method further includes the step of. adaptive^ specifying the respective processing times for 
each of the first and second states, wherein the processing time for the second state changes responsive to the 
processing of a pretfictrvefy encoded Macrobiock and a rjirfirectionally precfictivefy encoded Macrobiock. 

13. A method according to claim 12, wherein a predetermined amount of time is allocated for decoding a single image 
of the MPEG-2 data signal and the step of adaptively specifying the respective 

and second states specifies respective first and second portions of the predetermined amount of time as the 
respective processing times for the first and second states. 

14w A method accoiding to claim 11, further 

tor each of the state types, wherein the processing time for each state changes responsive to changes in the Mac- 
robiock decoding means. 
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